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Abstract 

A vehicle mounted control apparatus includes voice recognition section for recognizing a 
voice command input by an voice input device; and a control section that analyzes a cause of 
incapability of recognition of the voice command when the voice command cannot be 
recognized by the voice recognition section and gives a notice on result of the analysis. 
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TITLE OF THE INVENTION 
VEHICLE MOUNTED CONTROL APPARATUS 



BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 

[OOOl] The present invention relates to a vehicle mounted control apparatus for 

performing a control in accordance with a voice command. 



DESCRIPTION OF THE RELATED ART 

[0002] There has conventionally been provided a car navigation apparatus that 

outputs a conversational operating instruction by means of a screen or a voice to assist the 
key entry of a command by a user (for example, patent document 1). 

[0003] Moreover, a car navigation apparatus that can directly receive a command by a 

voice in a command execution mode while it does not display an operating procedure, has 
also been provided as a method capable of entering a command more quickly by entering a 
conversational command. 

[0004] [Patent document] Japanese Unexamined Patent Publication No. 2000-9771 9 

[0005] However, when a command by a voice (hereinafter referred to as "voice 

command' 1 ) is entered to a car navigation apparatus, depending on user's vocal conditions (for 
example, the level of voice uttered by a user), the car navigation apparatus cannot recognize 
the voice in some cases. In these cases, the user cannot understand which of the following 
reasons makes it impossible for the car navigation apparatus to recognize the voice and hence 
is embarrassed: a voice command is improperly issued; vocal conditions are not good; and 
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further, the apparatus is not operated because a switch for performing voice recognition is not 
pressed. 

[0006] Therefore, it is an object of the present invention to provide a vehicle mounted 
control apparatus that can inform a user about the state of recognition of a voice command 
uttered by the user in such a way that the user can easily understand the state of recognition. 

SUMAMRY SUMMARY OF THE INVENTION 

[0007] A vehicle mounted control apparatus of the present invention includes voice 

recognition section for recognizing a voice command that a user inputs by means of voice 
input device connected to outsid e; and a control section that analyzes a cause of incapability 
of recognition of the voice command when the voice command cannot be recognized by the 
voice recognition section and changes the display format of a scr ee n of display d e vic e 
conn e ct e d to outsid e on th e basis of r e sults gives a notice on result of the analysis. 
[0008] By this arrangement the control apparatus can provide the user with an 

occasion of practice of operation when the user wants to know an operating method and 
makes the user acquire the operation effectively. Moreover, the vehicle mounted control 
apparatus can also finish learning a parameter for recognizing the voice of the user through 
the practice for acquiring this operating method. Hence, it is possible to produce an effect of 
improving the rate of voice recognition and decreasing a work of the user even if the user 
does not conduct an additional work of inputting voice for the vehicle mounted control 
apparatus to be learned about voice of the user. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0009] Fig. 1 is a function block diagram to show a vehicle mounted control 
apparatus in accordance with embodiment of the present invention. 

[0010] Fig. 2 is a perspective view to show the driver's seat of a vehicle on which is 
mounted the vehicle mounted control apparatus in accordance with embodiment of the 
present invention. 

[0011] Fig. 3 is a flow chart to show the whole processing of the vehicle mounted 
control apparatus in accordance with embodiment of the present invention. 
[0012] Figs. 4 A, 4B are illustrations to show the display content of a display device in 
the vehicle mounted control apparatus in accordance with embodiment of the present 
invention. Fig. 4A illustrates the display content in which is displayed an icon for displaying 
the state of reception of voice and Fig. 4B illustrates the display content by which is 
displayed providing the user with the state of operation. 

[0013] Fig. 5 is an illustration to show the display content of the display device in the 
vehicle mounted control apparatus of embodiment of the present invention and illustrates a 
change in the display mode of a screen. 

[0014] Figs. 6 A, 6B are illustrations to show the display content of a display device in 
the vehicle mounted control apparatus in accordance with embodiment of the present 
invention. Fig. 6A illustrates the display content to provide the user with a menu capable of 
providing guidance on an operation and Fig. 6B illustrates the display content to provide the 
user with the content of a selected operation. 

[0015] Fig. 7 is an illustration to show the correspondence between analysis result 
and display format which is stored in memory of the vehicle mounted control apparatus of 
embodiment of the present invention. 
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[0016] Fig. 8 is an illustration to show the correspondence between the content of 
voice command and their example sentences which are stored in memory of the vehicle 
mounted control apparatus of embodiment of the present invention. 

BEST MODE FOR CARRYING OUT THE INVENTION 
[0017] In order to describe the present invention in more detail, the best mode for 

carrying out the present invention will be hereafter described with reference to the 
accompanying drawings. 
[0018] Embodiment 1 

[0019] Fig. 1 shows a vehicle mounted control apparatus of this embodiment 1 and a 

car navigation apparatus is here shown as one example of the apparatus. In Fig. 1 , a main 
device 1 includes a control section 2 constructed of a microprocessor or the like, a memory 3 
that is connected to the control section 2 and stores programs and various kinds of data, a 
map information storage section 4 that is also connected to the control section 2 and stores 
the digital data of maps, and a voice recognition section 5 (voice recognition section) that 
recognizes voice signals entered from a microphone 8 (voice input device). The control 
section 2 has a function that controls a display device 6 connected to the main device 1 and 
displays route information and road information necessary for navigation on the display 
device 6. As for the display device 6, a liquid crystal display device is generally used but all 
types of display devices can be used. The display device 6 may be integrated with the main 
device 1 or may be integrally embedded in a portion of interior surface of a vehicle. 
[0020] The control section 2 calculates the position of this apparatus itself by a 

calculation method using a publicly known technology from signals that are sent by a 
plurality of satellites and received by a GPS (Global Positioning System) antenna 7. The 
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microphone 8 (voice input device) converts user's voice into an electric signal and outputs the 
electric signal to the voice recognition section 5. The speaker 9 outputs voices, effect sounds, 
and sounds of music on the basis of the control of control section 2. 

[0021] The interface 10 has a function of relaying the operational status signals of 

electronic devices of the car and control signals to these electronic devices, for example, a 
control device of an air conditioner, head lights, and sensors for detecting the on-off states of 
a wiper and the head lights (all of which are not shown), between the control section 2 of the 
apparatus and these electronic devices. 

[0022] The input device 1 1 is a device to detect a command from a user and to output 

a signal corresponding to this command to the control section 2. As this input device 1 1 can 
be used various types of input devices capable of converting a user's command into a signal, 
for example, one or a plurality of buttons, a tablet, a touch sensor mounted on the display 
device 6, a joystick, and a lever mounted on a vehicle body. 

[0023] Fig. 2 is a perspective view of the driver's seat of a vehicle to which a vehicle 
mounted control apparatus in accordance with this embodiment is applied. 
[0024] A conversation starting switch 12 is a device for outputting to the control 

section 2 a signal to inform that the switch 12 is pressed by the user when it is pressed by the 
user.The microphone 8 is mounted in a position where driver's voice can be easily input, for 
example, on a dashboard forward of the driver's seat, and the conversation starting switch 12 
is mounted in a position where the user can easily press it during driving the vehicle without 
interfering with driving the vehicle, for example, on a steering wheel. 

[0025] This vehicle mounted control apparatus has two operation modes of a 

command execution mode and an operation guidance mode. 
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[0026] First, the command execution mode is a mode for performing an ordinary 
operation. In the command execution mode, when a user inputs voice to the apparatus, the 
voice recognition section 5 recognizes a voice command from the user and the control section 
2 executes a processing corresponding to the voice command. By this command execution 
mode, it is possible to control various kinds of vehicle mounted electronic devices, for 
example, to set a destination and to start a route guidance in navigation, and to control air 
volume of an air conditioner and an audio device, and to control electronic devices provided 
outside the vehicle through the Internet, such as an electronic device for electronic mail and 
Internet ITS (Intelligent Transport Systems). 

[0027] On the other hand, the operation guidance mode is a mode for giving the user 
a description of a method for performing voice input and the user can recognize the operating 
procedure and voice command of the vehicle mounted apparatus without looking at an 
operating manual. At this point, switching between the command execution mode and the 
operation guidance mode can be performed, for example, by pressing a selector button or by 
selecting a menu displayed on the display device 6. 

[0028] Fig. 3 is a flow chart of control section 2 that shows the command execution 

mode and the operation guidance mode. First, when the vehicle mounted apparatus is started, 
an initial screen at startup is displayed (step ST1). The control section 2 displays a message 
of "Please select a user" on the initial screen and displays a list of a plurality of users 
previously registered in the memory 3. The user looking at this display selects his own name 
from on e us e r of the list and inputs to the input device 1 1 . The control section 2 detects a 
signal output by input to the input device 1 1 and identifies the user on the basis of this signal 
(step ST2). 
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[0029] Next, the control section 2 detects an input signal from the input device 1 1 or 
the conversation starting switch 12 and checks whether this input signal is a signal to select 
executing an operation guidance mode or a signal to select executing a processing of 
command execution mode (step ST3). Here, when the user presses down the conversation 
starting switch 12, the control section 2 executes the processing of command execution mode 
at steps ST54 to ST8. On the other hand, when the user selects the operation guidance mode, 
the control section 2 executes a processing of operation guidance mode at steps ST10 and 
ST11. 

[0030] First, the operation of vehicle mounted apparatus in the command execution 
mode will be described. 

[0031] First, the control section 2 waits for an input signal from the input device 11 
and/or the input of voice command from the microphone 8 (step ST4) and at the same time, 
as shown in Fig. 4A, displays on the screen a "voice reception state icon" 13 of "VOICE" to 
show a state of reception of voice command. 

[0032] When the user inputs voice to the microphone 8, the voice recognition section 

5 recognizes the input voice. At this time, the voice recognition section 5 reads a recognition 
parameter for the user identified at step ST2 from the memory 3 and recognizes the voice by 
the use of this parameter (step ST5). For example, in a case where the user desires to retrieve 
a near gas station, when the user utters a voice command of "Near gas station", a voice 
command "Near gas station" is input from the microphone 8. 

[0033] Next, the control section 2 identifies which command of a plurality of voice 

commands that can be acceptable to the apparatus is uttered by the user is input. If the 
control section 2 can identify the voice command input , the control section 2 determines that 
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the state of voice of voice command is good and executes a processing corresponding to the 
identified voice command (step ST7). 

[0034] Here, "Near gas station" is input by the user as the voice command, so the 

vehicle mounted apparatus recognizes the voice command and hence retrieves a near gas 
station and displays the result of retrieval. In this regard, as shown in Fig. 4B, in order to 
show the state of operation to the user. "Near gas station is now being retrieved" is displayed 
during the operation. 

[0035] On the other hand, when the control section 2 cannot identify the voice 

command input , the control section 2 determines that a state of voice for the voice command 
is not good and analyzes the reason of the state of voice not good (step ST8). At this point, 
the analysis of the reason state of voice not good will be below described in detail. Further, 
the control section 2 changes display on the screen of display device 6 (step ST9) on the basis 
of the analysis results of state of voice which is shown in Fig. 5 with slanted lines, for 
example, change of color (meaning will be described later) and again waits for a voice 
command to be input (step ST93). 

[0036] Here, changing display on the screen means displaying an icon corresponding 

to the state of voice, changing the color of an icon or the whole screen which are already 
displayed in response to the state of voice, blinking the icon or the whole screen which are 
already displayed (in this case, an interval of blinking or the pattern of blinking is changed in 
response to the state of voice), displaying the state of voice as character information, and 
further using these operations in combination. In this regard, any display method can be 
employed as far as the user can visually recognize the state of voice when the user looks at 
the screen of display device 6. 
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[0037] Finally, the control section 2 detects on the basis of an electric signal or the 
like from the input device 11 or a power supply key via the interface 10 whether or not the 
user performs an operation of finishing a processing of vehicle mounted apparatus (step 
ST 12). At this point, when the user does not perform the operation of finishing, the control 
section 2 repeats the processing following step ST3. On the other hand, when the user 
performs the operation of finishing, the processing is finished. Here, "Near gas station" as 
the voice command is input by the user, so the vehicle mounted apparatus recognizes the 
voice command and hence retrieves a near gas station and displays the result of retrieval. In 
this regard, as shown in Fig. 4B, in order to show the state of operation to the user, "Near gas 
station is now being retrieved" is displayed during the operation. 

[0038] Next, the operation of vehicle mounted apparatus in the operation guidance 
mode will be described. At this point, for the sake of convenience in description, a case 
where a facility is retrieved so as to set a destination will be described as an example. 
[0039] When the user selects the operation guidance mode, the control section 2 
displays an initial screen on the display device 6 (step ST 10) as shown in Fig. 6 A. This 
initial screen is a screen to provide the user with a menu capable of giving the user guidance 
on operation by the use of operation guidance mode. In this case, the operation guidance 
mode includes "basic operation", "address retrieval", "facility retrieval", and "periphery 
retrieval". 

[0040] Next, the control section 2 detects a command input to the input device 1 1 by 
the user, and when the command is "facility retrieval", the control section 2 displays an 
operation describing screen shown in Fig. 6B (step ST11). In this display of describing 
operation, the control section 2 reads from the memory 3 a method of uttering a voice 
command relating to the selected command and displays it on the display device 6. Since 
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"facility retrieval" is selected in this case, the control section 2 displays "Utter the names of 
the prefecture and the name of facility consecutively" as the method of uttering the voice 
command corresponding thereto , and receives input of voices command by the user . 
[0041] At this point the operation guidance mode is usually performed firstly when 

use of the control apparatus of the present invention is begun, and in such a case input of user 
name to be registered in the memory 3 is performed through the input device 1 1 at step ST2, 
and the control section 2 stores the voice command input at the above described operation 
guidance mode by the user in memory 3 as a specific standard pattern of the voice 
recognition parameters for the registered user. 

[0042] Moreover, the control section 2 reads from the memory 3 an example sentence 

of a voice command relating to the command now selected and displays the example sentence 
on the display device 6. Since "facility retrieval" is selected in this case, "XX Zoo, Tokyo", 
"YY Station, Kanagawa Prefecture" and "ZZ Land, Chiba Prefecture" are displayed as 
example sentences. Here, it is also recommended that the displayed sentence is output as 
voice from the speaker 9 to make the user recognize the sentence as voice. 
[0043] In this regard, it is also recommended that when the control section 2 detects 

"to look at demonstration" as a command input to the input device 1 1 by the user, as shown in 
Fig. 6B, general operation including the operation of input device 1 1 (for example, "to press 
the conversation starting switch" at the time of inputting voice) be described. That is, in this 
case, when the user utters "the name of prefecture" and "the name of facility", the user needs 
to press the conversation starting switch 12 and hence in order to make the user know that he 
needs "to press the conversation starting switch at the time of inputting voice", it is visually 
displayed. 

[0044] Next, the operation of analysis of the state of voice will be described. 
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[0045] Assume that the user issues a voice command while pressing the conversation 

starting switch 12. When the control section 2 detects that the conversation starting switch 12 
is pressed down, the control section 2 records voice received by the microphone 8 in the 
memory 3 while detecting the switch 12 continues to b e b e ing pressed down. 
[0046] Next, when the control section 2 finish e s d e t e cting detects finish of detection 

that the conversation starting switch 12 is pressed down, the control section 2 finishes 
recording the voice in the memory 3 and instructs the voice recognition section 5 to analyze 
the recorded voice. The voice recognition section 5 analyzes the features of voice according 
to a publicly known voice recognition algorithm and records the analysis results in the 
memory 3. For example, the voice recognition section 5 compares the standard pattern of 
voice of the user previously stored in the memory 3 with the pattern of input voice to analyze 
a plurality of features, for example, sound level, uttering speed, uttering timing, and whether 
or not unnecessary words (hereinafter referred to as "incidental word") are included and 
outputs analysis results. 

[0047] Next, the control section 2 determines whether or not the state of input voice is 

good on the basis of analysis results. Any parameter which shows the ease of voice 
recognition, can be used to determine whether or not the state of input voice is good. For 
example, when it is detected that any one of sound level, uttering speed, uttering timing, and 
the presence or absence of incidental words is "not good", the control section 2 determines 
that total analysis result is "not good". On the other hand, when all the analysis results for 
each item of analysis are "good", the control section 2 determined that total analysis result is 
"good". 

[0048] Here, when the control sections 2 determines that the analysis result is good, 

as described above, the control section 2 executes the recognized voice command. On the 
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other hand, when the control sections 2 determines that the analysis result is "not good", as 
described above, the control section 2 changes the display of screen on the basis of display 
format associated with the analysis result. The analysis result corresponds to the display 
format in a manner shown in Fig. 7 and this correspondence is stored in the memory 3. For 
example, when the control section 2 recognizes that the analysis result is "not good" because 
"sound level is high", the control section 2 refers to a state-of-voice correspondence table 
stored in the memory 3 and recognizes that a color corresponding to a case of "sound level is 
high" is "blue". Further, the control section 2 changes the color of an icon displayed on the 
screen, for example, the color of the above described icon 13 of "VOICE" to show the state of 
reception of voice, is changed "blue" in this case as te-a state shown in Fig. 5 with slanted 
lines and again waits for an input of command by voice. At this point, the icon on which the 
change of display format is performed, is not limited to the icon 13 of "VOICE" to show the 
state of reception of voice but may be an icon to show the present position. Further, the 
whole screen may be changed to blue. 

[0049] Here, in the analysis results, "sound level is high" and "sound level is low" are 
the analysis results from a viewpoint of "sound level", and "conversation speed is high" and 
"conversation speed is low" are the analysis results from a viewpoint of "conversation speed", 
and "conversation timing is early" and "conversation timing is late" are the analysis results 
from a viewpoint of "conversation timing". Hence, as for a color corresponding to the 
analysis results from the same viewpoint, a color of the same group may be used. That is, as 
described above, "sound level is high" and "sound level is low" are the analysis results from a 
viewpoint of "sound level", so they are displayed in a color of the same group, for example, 
"sound level is high" is displayed in "blue" and "sound level is low" is displayed in "greenish 
sky blue". Further, when a level is higher than a standard, for example, "high" and "early", a 
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dark color is used and when a level is lower than a standard, for example, "low" and "late", a 
light color is used. With this, the user understands what is wrong from the group of color and 
can get a sense of "higher than a standard" or "lower than a standard" from the shades of 
gray. 

[0050] Next, an analysis of voice will be described in detail. 

[0051] For example, whether the input voice is good or not is determined by the 

following method. 

[0052] (1) Presence or absence of incidental word at the beginning of sentence 

[0053] The voice recognition section 5 compares the standard pattern of an incidental 
word stored in the memory 3 with the input voice and produces an output to the effect that 
analysis result is "not good" when the beginning of voice is in good agreement with the 
standard pattern of incidental word. On the other hand, when the voice does not have any 
incidental word at the beginning of voice, the voice recognition section 5 produces an output 
to the effect that analysis result is "good". 
[0054] (2) Sound level 

[0055] The voice recognition section 5 detects whether or not the sound level of input 

voice is within a predetermined range and produces an output to the effect that analysis result 
is "not good" when the sound level is not within the predetermined range. On the other hand, 
when the sound level is within the predetermined range, the voice recognition section 5 
produces an output to the effect that analysis result is "good". The predetermined range is set 
at a range of less than 0 dB to -30 dB when the maximum value of sound level to be detected 
by the microphone 8 is 0 dB. 

[0056] (3) Length of conv e rsation tim e Conversation pace 
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[0057] The voice recognition section 5 measures the length of time of the input voice 

and compares this length of time with the length of time of the standard pattern and 
determines that this l e ngth of tim e the conversation pace is good when a difference between 
them is within a predetermined range. 

[0058] This predetermined range can be arbitrarily set and, for example, a difference 

of the length of time of the input voice can be set at a range of +25 % to -25 % of the length 
of time of the standard pattern. 
[0059] (4) Uttering timing 

[0060] The voice recognition section 5 determines whether or not the uttering timing 

of input voice is within a predetermined range of voice acceptable timing . At this point, the 
voice recognition section 5 produces an output to the effect that analysis result is "good" 
when the uttering timing is within the predetermined range. On the other hand, when the 
uttering timing is not within the predetermined range, the voice recognition section 5 
produces an output to the effect that analysis result is "not good". For example, when voice 
has been begun to input at the beginning of a period during which the microphone 8 can 
acquire the voice, the voice recognition section 5 produces an output to the effect that 
analysis result is "not good". On the other hand, in cases where the input voice has not yet 
completed to utter for a period more than the maximum input time of voice command or 
where a voice input is detected only at the end of the above described period during which 
the microphone 8 can acquire the voice, the voice recognition section 5 produces an output to 
the effect that analysis result is "not good". In the cases other than these cases, the voice 
recognition section 5 produces an output to the effect that analysis result is "good". 
[0061] (5) Presence or absence of incidental word at the end of sentence 
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[0062] As is the case with the detection of an incidental word at the beginning of a 
sentence, the voice recognition section 5 detects whether or not there is an incidental word at 
the end of a sentence. When the voice recognition section 5 detects an incidental word, the 
voice recognition section 5 produces an output to the effect that analysis result is "not good' 1 . 
On the other hand, when the voice recognition section 5 does not detect an incidental word at 
the end of sentence, the voice recognition section 5 produces an output to the effect that 
analysis result is "good". 

[0063] Next, a set of example sentences will be described in detail. Fig. 8 is a list in 
which the contents of voice commands are correlated with example sentences that the user 
utters so as to execute the respective voice commands. 

[0064] The sentences of the contents (a) to (e) of voice commands, shown in this 

drawing, include a plurality of commands of different diction and the voice recognition 
section 5 can recognize voice commands on the basis of a plurality of voices uttered by the 
different diction. 

[0065] In this regard, while the above description has been given with a particular 

emphasis on the car navigation apparatus, the vehicle mounted control apparatus of the 
present invention is not limited to this example but the present invention can be applied to 
any type of vehicle mounted control apparatus for operating electronic devices inside and 
outside of the vehicle. For instance, the vehicle mounted control apparatus includes of a 
control apparatus for controlling vehicle mounted electronic devices, for example, an air 
conditioner, an audio unit and the like. Moreover, as a control apparatus for controlling 
electronic devices outside of the vehicle may be thought, for example, a control apparatus for 
controlling various kinds of electronic devices outside of the vehicle through a transmitter 
which is connected to the interface 10. The electronic devices outside of the vehicle include 



16 Attorney Docket No.: 1 163-05 15PUS1 
SPECIFICATION (MARKED-UP VERSION) 

all kinds of electronic devices which are connected by a communications lines, for example, 
an air conditioner intended for home use or commercial use, a home security system, a home 
server, other electric appliances, an automated paying machine installed at a store such as a 
fast food store or a gas station, and a gate installed at the entrance and exit of a parking lot. 
[0066] Further, in the control apparatus in Fig. 1, while an example has been shown 
as the voice recognition section 5 which is constructed of an LSI for voice recognition, the 
voice recognition section 5 is not limited to a case where it is constructed of a hardware 
circuit designed exclusively for voice recognition but may employ a software program for 
voice recognition. In this case, the program for voice recognition is executed by the control 
section 2 or a separate processor used exclusively for voice recognition. 

[0067] Still further, in the above embodiment, while an example of the voice 

recognition processing is given in which the voice recognition section 5 performs voice 
analysis as a preliminary processing and the control section 2 executes a processing of 
displaying advice by use of the result of analysis, but the control section 2 can executes also 
the voice analysis itself . 

[0068] Still further, in the above described embodiment, while the recognition 

parameters of a plurality of users are registered, because there are also cases where only one 
specified user uses the control apparatus, it is not always necessary to classify the users when 
there is no specific need . In a case where only one specified user uses the control apparatus, 
a processing of registering users at step ST2 in Fig. 3 is not required. 

[0069] As for the memory 3, a volatile and/or nonvolatile memory can be used. 
Moreover, a storage device such as a hard disk and DVD-RAM can be used as storage 
device. 
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[0070] Industrial Applicability 

[0071] As described above, a vehicle mounted control apparatus in accordance with 
the present invention is applicable to a car navigation apparatus. 



